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(57) The precision of an initial image is reduced 
through image automatic processing in a man- 
ner simulating techniques used in the produc- 
tion of a custom art work. The resulting image, 
although generated without human interven- 
tion, has a pleasing, customized appearance 
which can be desirable to consumers seeking 
decorative art work. One way of reducing the 
precision of the initial image is modifying por- 
tions of the image to have more uniform lumi- 
nance, color saturation, or color hue, producing 
a resulting image appearing to contain brush 
strokes. Alternatively, the precision of the initial 
image is reduced by locating contrast edges in 
the image, and modifying the luminance along 
these edges, producing a resulting image in 
which objects and features are outlined and 
have an enhanced definition from the back- 
ground. 
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The present invention relates to the automatic 
generation, e.g. for display or printing, of images 
which resemble custom artwork. 

It is known to apply computers to problems of im- 
age generation and enhancement. Image enhance- $ 
ment techniques have been applied to scientific ap- 
plications, producing higher definition images provid- 
ing improved insight. Computer animation has been 
used to produce high-definition images, resembling 
photographs, for movies and other entertainment me- 10 
dia. As computers and software have become more 
powerful, computer generated or enhanced images 
have become closer to the photographic ideal. 

Some software packages also provide special ef- 
fects which deform images. For example, some pack- 1 s 
ages can modify the color or luminance of an image 
to produce what appears to be a photographic nega- 
tive of the image. 

Some consumers seeking decorative artwork 
prefer custom art work such as oil and watercolor 20 
paintings or lithography to more precise art forms 
such as photography. Custom art work includes an 
expressive aspect often lacking from photography; 
typically the colors and important elements of custom 
art work are emphasized and exaggerated as com- 25 
pared to a photograph, which can result in a pleasing 
overall impact. Furthermore, the customized nature 
of custom art work is often immediately apparent from 
the coarseness of presentation, e.g., from the brush 
strokes and object outlines which can be seen upon 30 
close inspection; this customized appearance can 
also enhance a consumer's perception and attraction 
to a work. 

The drive in automated computer image process- 
ing has been toward either (1) enhancing the preci- 35 
sion of an images to approach a photographic ideal, 
or (2) modifying an image to produce a manifestly 
computer-generated result 

An embodiment of the present invention descri- 
bed hereinbelow seeks a different goal. In accor- 40 
dance with the embodiment of the invention, the pre- 
cision of an initial image is reduced through automatic 
image processing in a manner simulating techniques 
used in the production of a custom art work, produc- 
ing an image with a pleasing, customized appearance 45 
which can be desirable to consumers seeking decor- 
ative art work. Although the resulting image appears 
to be the product of human labor, the process does 
not require human intervention. 

Respective different aspects of the inventions so 
are set out in claims 1 and 24 hereof. 

According to another aspect of the present inven- 
tion, the precision of an initial image is reduced by 
modifying portions of the image to have more uniform 
luminance, color saturation, or color hue, producing a 55 
resulting image appearing to contain brush strokes 
similar to those in oil paintings. 

Specifically, for each portion simulating a brush 



stroke, luminance and color values are selected from 
one pixel in the initial image and used as the basis for 
ail of the pixels in the portion. To enhance the overall 
contrast of the image, the luminance of the selected 
value may be reduced if the value is less than a com- 
puted average, otherwise the luminance of the select- 
ed value may be increased. The pixels of the. portion 
are then generated from the computed luminance and 
color values. A small random variation or dithering 
may also be added to produce a more realistic result. 

The portions may be elongated, and in one em- 
bodiment have parabolic curved elongated edges, 
and thus have a more realistic brush-stroke-like ap- 
pearance. These elongated portions may be ar- 
ranged in one of four orientations (vertical, horizontal, 
or diagonal); the orientation may be determined by 
measuring the direction of maximum contrast in a 
blurred version of the initial image, and orienting the 
portion perpendicular to this direction, simulating typ- 
ical painting techniques. For further improved simu- 
lation, the size of the portion may be modified in re- 
sponse to the level of contrast The portions thus cre- 
ated may be overlaid on a background formed by blur- 
ring the initial image. 

To further improve the simulation, smaller por- 
tions simulating small brush strokes may be created 
over the initial set of large portions in regions of high 
contrast These smaller portions simulate detailed 
brush strokes usually created near object edges in an 
original oil painting. 

In another aspect, the precision of the initial im- 
age may be reduced by locating contrast edges in the 
image, and modifying the luminance along these 
edges, producing a resulting image in which objects 
and features are outlined and therefore have an en- 
hanced definition from the background, such as in 
watercolor paintings. . 

Specifically, a differential measurement of con- 
trast may be generated at each pixel of the initial im- 
age; then, after processing these contrast measure- 
ments with a logarithmic function, the contrast meas- 
urements may be combined with the initial image, en- 
hancing the edges in the initial image. For maximum 
enhancement, edges in bright regions are brightened, 
whereas edges in dark regions are darkened. 

The embodiment of the invention disclosed here- 
inbelow enables the generation of imitation custom 
artwork by simulating brush strokes and/or enhanc- 
ing edges. 

The invention will now be further described, by 
way of illustrative and non-limiting example, with ref- 
erence to the accompanying drawings, in which: 
Figs. 1Aand 1B are perspective views of a video 
printer, video camera, and monitor which can util- 
ize principles of the present invention, and Fig. 
1 C is a functional block diagram of computing cir- 
cuitry cooperating with the printer, camera, or 
monitor; 
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Fig. 2A is an initial image prior to processing, Fig. 
2B is an image produced by generating simulated 
brush strokes in the image of Fig. 2A in accor- 
dance with an embodiment of the invention, and 
Fig. 2C is an image produced by detecting and 
enhancing edges in the image of Fig. 2Ain accor- 
dance with an embodiment of the invention; 
Fig. 3 is an overall flow chart of a computer pro- 
gram for adding simulated brush strokes to an im- 
age; 

Fig. 4 is a flow chart of a procedure for reading a 
stored file for an image; 

Fig. 5 is an illustration of the relationships of Sat- 
uration (S) and Hue (H) color values to U and V 
color values; 

Fig. 6 is a flow chart of a procedure for converting 
U and V color values to S and H values; 
Fig. 7 is a flow chart of a procedure for painting 
brush strokes onto an image; 
Fig. 8 is a flow chart of a subprocedure for deriv- 
ing brush stroke sizes and orientations; 
Fig. 9 is a flow chart of a subprocedure for filling 
out a brush stroke on the image; 
Figs. 10A, 10B, 10C and 10D are illustrations of 
brush strokes formed by the subprocedures of 
Figs. 8 and 9; 

Fig. 11 is a flow chart of a procedure for convert- 
ing S and H color values to U and V values; 
Fig. 12 is a flow chart of a computer program for 
enhancing edges of an image; 
Fig. 13 is a flow chart of a procedure for finding 
edges in an image by computing differential lumi- 
nance values; 

Figs. 14Aand KB are illustrations of the manner 
in which differential luminance values are com- 
puted by the procedure of Fig. 13. 
Fig. 15 is a flow chart of a procedure for modify- 
ing the output of the procedure of Fig. 1 3 to create 
a more realistic distribution of differential values; 
Figs. 16Aand 16B are histograms of differential 
values respectively before and after modification 
by the procedure of Fig. 15; 
Fig. 1 7 is a flow chart of a procedure for combin- 
ing differential values with an initial image; and 
Fig. 18 is a flow chart of a procedure for smooth- 
ing a combined image to produce a result. 
Fig. 1A illustrates a video printer 10. The printer 
1 0 may be coupled to a video camera 12 via shielded 
cables 1 4. Printer 10 includes a control panel 16 hav- 
ing buttons 18 with which an operator can control vid- 
eo printer 10 to select a desired image on a video tape 
in camera 12. When a desired image has been locat- 
ed, the operator presses button 20 to generate a pho- 
tograph 22 having a still print of the selected image. 
Video printer 10 may be used, for example, by con- 
sumers to produce still photographs from videotapes 
of vacations, family gatherings, etc. Thus, video prin- 
ter 10 allows video camera 12 to serve the dual roles 



of a video and still camera. 

Fig. 1 B illustrates a monitor 23 which can be sim- 
ilarly coupled to video camera 12 via cable 14, to dis- 
play video from camera 12. Alternatively, any other 
5 video source, such as a television broadcast, a desk- 
top VCR, or a laser disc player, can provide video to 
monitor 23. 

Referring to Fig. 1C, in accordance with princi- 
ples of the present invention, computing circuitry 25, 

10 e.g. special purpose computing circuitry and/or one 
or more microprocessors running software, process- 
es images before the images are printed by printer 10 
or displayed on monitor 23. 

As described in detail below, the processed irrv 

15 age has characteristics simulating the appearance of 
original artwork. Thus, when printer 10 is used with 
computing circuitry 25, printer 10 generates still im- 
ages simulating custom artwork; when monitor 23 is 
used with computing circuitry 25, monitor 23 gener- 

20 ates either still images simulating custom artwork, or 
a series of sequentially-displayed images, which sim- 
ulate custom animation. 

Computing circuitry 25 may be mounted in a sep- 
arate unit 25 as illustrated in Fig. 1 C, or may be incor- 

25 porated into the existing circuitry of printer 10, moni- 
tor 23 or camera 12. 

Referring to Fig. 2A, there is shown an unpro- 
cessed image of the type which could be produced by 
printer 10 or monitor 23. Indicated on the image are 

30 two edges of contrast 24, 26 and areas 28 of varying 
contrasts. 

Referring to Fig. 2B, in accordance with an em- 
bodiment of the present invention, the image of Fig. 
2A may be modified to simulate techniques used in 
35 producing oil paintings. As described in greater detail 
in the following figures, to perform such a modifica- 
tion, portions of the initial image are modified to have 
more uniform luminance color saturation and/or color 
hue, producing visible portions of uniform luminance 
40 or hue in the resulting image that resemble a brush 
strokes of oil paintings. Such simulated brush strokes 
29 can be seen in areas 28 in Fig. 2B. Furthermore, 
as described in more detail below, the overall contrast 
of the image is increased, as is apparent by compar- 
es ison between Figs. 2B and 2A. 

Referring to Fig. 2C, in accordance with a prefer- 
red feature of the invention, the initial image shown 
in Fig. 2Amay be modified by locating contrast edges 
in the image, such as edges 24 and 26, and modifying 
50 the luminance of the image along those edges, pro- 
ducing a resulting image in which contrast lines such 
as lines 24 and 26 are significantly more pronounced, 
simulating the appearance of watercolor original art- 
work. 

55 The following figures describe procedures for 

producing the effects described above. The proce- 
dures described herein were used to generate, on a 
computer display, the images illustrated in Figs. 2B 



5 



EP 0 678 831 A2 



6 



and 2C. 

Although the following description details a pro- 
gram for a general purpose computer, the image proc- 
essing procedures described below can be carried 
out in various other ways, e.g. by software running on 
a microprocessor embedded in a special purpose de- 
vice, such as video printer 10, monitor 23, camera 12, 
or a stand-alone image processing unit Alternatively, 
these procedures can be performed by special pur- 
pose computing circuitry in an expansion slot of a 
general purpose computer or embedded in a special 
purpose device. For clarity, in the following descrip- 
tion, the general term "computing circuitry'* will be 
used to collectively refer to any electronic configura- 
tion which can perform a computation, including by 
one or more microprocessors running software, by 
special purpose computing circuitry, or by a combin- 
ation of both. 

Referring to Fig. 3, to produce simulated brush 
strokes on an initial image, the computing circuitry 
first reads 30 a stored input file containing a pixellated 
representation of the image to be processed. Next, 
the image is converted 32 into a representation in 
which it may be more easily processed (as discussed 
with reference to Fig. 5 below). Then, the computing 
circuitry steps through the pixels of the image, pro- 
ducing or "painting" 34 simulated brush strokes onto 
the image. Once the image has been fully painted, 
the computing circuitry converts 36 the color repre- 
sentation back to its initial format, and writes 38 an 
output file containing the processed image. 

Referring to Fig. 4, to read an input file 30, the 
computing circuitry first reads 31 luminance (Y) val- 
ues from input file and stores these values in a work- 
ing buffer. Then the computing circuitry creates 33 a 
working copy of the image by subtracting a predeter- 
mined offset from each Y value and limiting the result 
to an allowable range. Thereafter, the computing cir- 
cuitry reads 35 U color values, and reads 37 V color 
values from the input file and stores these values in 
working buffers. 

Referring to Fig. 5, the relationship between U 
and V color values and saturation (S) and hue (H) col- 
or values are based on trigonometry. If the U and V 
values represent the length of the two legs of a right 
triangle, the length of the hypotenuse of this triangle 
represents the color saturation value S, and the angle 
adjacent to the U leg represents the color hue value 
H. 

Thus, referring to Fig. 6, to convert 32 color val- 
ues to a usable representation at each pixel in the 
working image, the computing circuitry computes 39 
the saturation value using the Pythagorean theorem, 
and computes the hue value from the inverse tangent 
of the ratio of the V value and the U value. 

Referring to Fig. 7, to paint 34 brush strokes onto 
the image in the working buffer, the computing circui- 
try first blurs 41 the initial image by replacing each 



pixel's luminance value with the average of the lumi- 
nance values of, e.g., 100 pixels in a square 10 by 10 
pixel surrounding the pixel. This blurring process fills 
in gaps between brush strokes to be added by the 
5 procedures described below, produce a more realistic 
result 

After the image in the working buffer has been 
blurred, the computing circuitry computes 42 the 
average luminance of the central one-third of the im- 

10 age. This value is used in the subsequent processing 
to enhance the contrast of the image, resulting in a 
more realistic result. 

After the proceeding preliminary processing, the 
computing circuitry begins to generate simulated 

15 brush strokes. To do so, the computing circuitry first 
creates a working buffer to containing luminance, sat- 
uration and hue values for each pixel of the resulting 
image. The computing circuitry then fills this buffer 
with the blurred luminance, saturation and hue values 

20 from the initial image produced in step 41 described 
above. Because the buffer is initialized with a blurred, 
background version of the initial image, any gaps be- 
tween simulated brush strokes are not noticeable in 
the resulting image. 

25 To produce brush strokes, the computing circui- 

try scans three times through the image in blocks of, 
e.g., 7 pixels on a side. During the scans, the comput- 
ing circuitry produces simulated brush strokes in vis- 
ited blocks. During the first two scans 44, the comput- 

30 ing circuitry produces large brush strokes in every 
visited block; during a third scan 55. the computing 
circuitry produces small brush strokes in visited 
blocks which meet certain criteria, as elaborated be- 
low. 

35 The large brush strokes may be larger than the 

size of the block, and thus may overlap with brush 
strokes of neighboring blocks; when large brush 
strokes overlap, the more recent brush stroke erases 
the overlapping portions of the less recent brush 

40 stroke. To enhance the realism of the result large 
brush stroke scanning 44 is interleaved; the f irst scan 
is performed along regularly-spaced rows of blocks 
crossing the image, and the second scan is per- 
formed along regularly-spaced rows which are inter- 

45 leaved with the rows of the first scan. By performing 
two interleaved scans, the resulting image includes a 
realistic mix of complete brush strokes (from the sec- 
ond scan) and partially erased brush strokes (from 
the first scan). 

so At each block during the three scans, the comput- 

ing circuitry first inspects 45 the image in the working 
buffer and derives a brush stroke size and a brush 
stroke orientation. Brush stroke sizes and orienta- 
tions are generated from the blurred image rather 

55 than the original image. This is done because the 
blurred image is relatively uncontaminated by noise 
and therefore will produce a smoother and more reg- 
ular pattern of brush stroke sizes and orientations, 



7 



EP 0 678 831 A2 



8 



which results in a more realistic effect. 

To create a brush stroke, the computing circuitry 
selects 46 the luminance, saturation and hue values 
for the pixel at the center of the new brush stroke, and 
computes a new luminance and saturation value by 
dithering 48 (i.e., adding a randomly generated posi- 
tive or negative increment to) the luminance and sat- 
uration values from pixel at the center of the brush 
stroke. 

Next, to increase the overall contrast of the im- 
age, the computing circuitry compares 50 the com- 
puted luminance value to that from the central one- 
third of the image computed in step 42. If the comput- 
ed luminance is less than that of the central one-third 
of the image, the computed luminance is decreased; 
however, if the computed luminance is greater than 
that of the central one-third of the image, the comput- 
ed luminance is increased. 

Finally, the brush stroke is filled out 52 with the 
computed luminance, saturation and hue values and 
the computing circuitry proceeds to the next step 
across the image and repeats at step 45. 

As noted above, during the first two scans, large 
brush strokes are generated in every visited block; 
thus the four steps 46, 48, 50 and 52 and performed 
in every visited block. However, during the third scan, 
small brush strokes are created in only a few of the 
visited blocks. 

In an original oil painting, object edges are typi- 
cally defined by a relatively large number of small 
brush strokes, a natural result of the need for preci- 
sion when painting such edges. However, the body of 
an object is typically defined by a smaller number of 
large brush strokes, because less precision is needed 
in these areas. This methodology is simulated in ac- 
cordance with the invention by producing small brush 
strokes only along contrast lines in the image. 

Specifically, during the scanning 55 which gener- 
ates small brush strokes, the computing circuitry de- 
termines 57 at each block whether the image has high 
contrast at the location of the block, and only gener- 
ates a small stroke if the image has high contrast. As 
elaborated below with reference to Fig. 8, inspection 
step 45 measures the contrast of the image at the cur- 
rent block and produces a small brush stroke size if 
the image has high contrast. Therefore, at step 57 the 
computing circuitry determines whether the brush 
stroke is small, e.g. less than seven pixels long, and 
if so generates the brush stroke through steps 46, 48, 
50 and 52. If the brush stroke is larger than seven pix- 
els, there are no significant lines of contrast in the 
current block, and the computing circuitry proceeds 
to the next block without generating a small brush 
stroke. 

Referring to Fig. 8, at step 45, the computing cir- 
cuitry inspects the image to derive brush stroke sizes 
and orientations. This is done by computing 47 the 
contrast of the initial image along four directions. 



These four directions can be, for example, vertical, 
horizontal and diagonal from lower left to upper right 
and diagonal from upper left to lower right. To com- 
pute the contrast in each of these directions, the com- 
5 puting circuitry computes the difference in luminance 
values of appropriate groups of pixels neighboring the 
pixel at the center of the brush stroke. For example, 
to compute the contrast in a horizontal direction, the 
computing circuitry computes the difference be- 
to tween luminance values at one or more pixels directly 
to the right of the central pixel and luminance values 
at one or more pixels directly to the left of the central 
pixel. The absolute value of the difference represents 
the contrast of the image at the brush stroke in the 
t5 horizontal direction. Similar computations are per- 
formed along vertical and diagonal directions to gen- 
erate four contrast values. The computing circuitry 
then determines which of these four values is the 
maximum value. The brush stroke is then oriented in 
20 a direction perpendicular to the direction of the max- 
imum contrast value. By orienting brush strokes per- 
pendicular to the direction of the maximum contrast, 
brush strokes in the processed image will be gener- 
ally aligned along rather than across contrast lines in 
25 the image. This enhances the simulation of oil paint- 
ing techniques, because oil paintings typically include 
brush strokes aligned along the direction of object 
outlines. 

After computing the brush stroke direction, the 
30 brush stroke length is computed 49 with reference to 
the maximum contrast level determined in the pro- 
ceeding step. Where the maximum contrast level is 
large, the length of the brush stroke is shortened. This 
also simulates oil painting techniques in that oil paint- 
35 ings typically include short brush strokes at detailed 
edges surrounding objects. The width of the brush 
stroke is computed 51 with reference to the length, so 
that the brush stroke is slightly longer than it is wide. 
Exemplary brush strokes are illustrated in Figs. 10A, 
40 10B, 10C and 10D, and are discussed in further detail 
below. 

Referring to Fig. 9, to fill out a brush stroke with 
computed luminance saturation and hue values 52 
the computing circuitry repeatedly performs 54 a ser- 
45 ies of steps along the length of the brush stroke. The 
brush strokes produced may be rectangular or leaf- 
shaped. 

Exemplary rectangular brush strokes are shown 
in Figs. 10Aand 10B. Fig. 10A shows a horizontally 

so oriented brush stroke having a horizontal length Land 
a vertical width W. Fig. 1 0B shows a diagonally orient- 
ed brush stroke of the same dimensions. (The hori- 
zontal and vertical dimensions in Fig. 10B are re- 
duced by a factor of the square root of two because 

55 the stroke is at a 45 degree angle.) 

Fig. 10C shows a leaf-shaped brush stroke in a 
horizontal orientation and Fig. 10D shows a leaf- 
shaped brush stroke in a diagonal orientation. The 
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leaf-shaped brush strokes are characterized by 
curved sides along their long dimensions. The curves 
are generated by a quadratic equation, and thus the 
curves are parabolic curves. The leaf-shaped brush 
stroke shown in Fig. 10C has a horizontal length L 
equal to the length L of the brush stroke shown in Fig. 
10A. The maximum width W of the leaf-shaped brush 
stroke shown in Fig. 10C is equal to the width of the 
brush stroke shown in Fig. 10A. Similar relationships 
can be found between the diagonally oriented leaf- 
shaped brush stroke shown in Fig. 10D and the diago- 
nally oriented rectangular brush stroke shown in Fig. 
1 0B. (Again, the horizontal and vertical dimensions in 
Fig. 10D are reduced by a factor of the square root of 
two because the brush stroke in Fig. 10D is at a 45 
degree angle.) 

As noted above with reference to Fig. 9, brush 
strokes are produced by a series of steps executed 
along the length of the brush stroke. During each 
step, the computing circuitry follows 58 a vertical, 
horizontal, or diagonal line of pixels, such as lines 59 
illustrated in Fig. 10A-10D, across the width of the 
brush stroke, replacing each pixel along the line with 
the computed luminance, saturation and hue values. 
The computing circuitry starts at a pixel at a randomly 
selected position near to the edge of the brush stroke. 
For increased realism, the computing circuitry adds 
slight randomness to the appearance of the image, by 
randomly varying the luminance value Y as a line 59 
is followed across the stroke. To do this, the lumi- 
nance value is varied based upon the current location 
across the width of the brush stroke. 

After filling out a line 59 across the width of the 
brush stroke, the procedure of Fig. 9 proceeds to the 
subsequent line across the width of the brush stroke, 
and so on until the brush stroke has been filled out 
completely. 

Leaf-shaped brush strokes are created by an ad- 
ditional step 56 performed during the above- 
described procedure. In this step the computing cir- 
cuitry computes 56 a brush stroke width based upon 
the current location along the length of the brush 
stroke. The equation used to compute the width value 
is, as noted above, a quadratic equation, and gener- 
ates a maximum width value at the center of the brush 
stroke. After the brush stroke width has been comput- 
ed, the computing circuitry fills out a line 59 across 
the brush stroke having the computed width, in the 
manner described above. 

Referring to Fig. 11, after the image has been 
processed to include brush strokes, the color repre- 
sentation is converted 36 back to U and V color val- 
ues. To do this, for each pixel in the working buffer, 
the computing circuitry computes 60 the saturation S 
multiplied by the cosine of the hue H to generate the 
U value, and computes the saturation S multiplied by 
the sine of the hue H to produce the V value. 

Referring to Fig. 12, a preferred feature of the in- 



vention provides procedures for converting an image 
to have the appearance of watercolor painting. This 
procedure is shown generally in Fig. 1 2 and more spe- 
cifically in the subsequent figures. 
5 Referring to Fig. 12, to perform an edge detection 

conversion to generate the appearance of watercolor 
painting, the computing circuitry first reads 70 the lu- 
minance values from an input file into two buffers. 
(Color values are not processed in this procedure, al- 
to though color processing, e.g., reducing or increasing 
color saturation, could be added to the procedure de- 
scribed herein to produce more realistic watercolor 
images.) The computing circuitry then computes 72 
the average luminance value of the central one-third 
15 of the image. This value is used in subsequent proc- 
essing described below. Next, the computing circuitry 
performs 74 a filtering operation on the first of the two 
buffers to locate contrast regions in the image. Vari- 
ous edge detecting filtering operations can be used 
20 at step 74, such as Laplacian filtering, Forsen filter- 
ing, and/or Sobel filtering. Sobel filtering has been 
found to create the strongest edges, and is described 
below. After filtering, the computing circuitry proc- 
esses 76 the luminance values in the first buffer by 
25 performing a logarithmic function on the values in the 
first buffer. After the above processing, the comput- 
ing circuitry combines 78 the first buffer with the sec- 
ond, unmodified buffer. During this operation, the 
computing circuitry uses the contents of the first buf- 
30 fer to create dark edges in dark areas of the original 
image in the second buffer, and vice-versa. The re- 
sulting image is smoothed 80 in a manner discussed 
below, producing a final image which is written 82 to 
an output file. 

35 To Sobel filter 74 an image in the first buffer, the 

computing circuitry performs 75 a sequence of steps 
shown in Fig. 1 3 for each pixel in the first buffer. First, 
the computing circuitry computes 77 a weighted dif- 
ferential in the horizontal direction. Then, the comput- 
40 ing circuitry computes 79 a weighted differential in 
the vertical direction. The absolute values of these 
differentials are then summed 81 and the result is lim- 
ited 83 to the range of permissible luminance values. 
Finally, the luminance values are reversed 85 from 
45 black to white and white to black, so that areas having 
high contrast are identified by relatively small lumi- 
nance values in the first buffer, and vice-versa. To re- 
verse white and black for the Sobel-f ilter result of a 
pixel, the minimum (black) and maximum (white) al- 
so lowable luminance values are added together, and 
' the result of the Sobel-filtering for the pixel is sub- 
tracted from the sum. If the Sobel-f ilter result for the 
pixel had a large (near-white) value, the result will be 
a proportionally small (near-black) value, and vice- 
55 versa. 

To compute the weighted differentials steps 77 
and 79 of Fig. 13, the computing circuitry multiplies 
neighboring pixels of the pixel being processed by 
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weighted factors shown diagrammatical! y in Fig. 14A 
and Fig. 14B. To compute a weighted differential in 
the horizontal direction, and thereby locate vertically- 
oriented lines of contrast, the computing circuitry 
multiplies the three pixels to the left, upper left and to 
the lower left, respectively, by the values -2, -1 and - 
1; then, the computing circuitry multiplies the three 
pixels to the immediate right, upper right and lower 
right of the pixel being processed, respectively, by the 
values 2, 1 and 1. The results of these six multiplica- 
tions are added together to produce a differential val- 
ue. 

Similarly, as shown in Fig. 14B, to compute a dif- 
ferential in the vertical direction, and thereby locate 
horizontally-oriented lines of contrast, the pixels im- 
mediately above, above and to the left, and above 
and to the right of the pixel being processed are re- 
spectively multiplied by -2, -1 and -1; and the pixels 
immediately below, below and to the left and below 
and to the right of the pixel being processed are mul- 
tiplied respectively by 2, 1 and 1 . The results of these 
multiplications are added together to generate a dif- 
ferential value in the vertical direction. 

As can be seen from the proceeding discussion, 
the horizontal and vertical differential values repre- 
sent the extent of horizontal and vertical contrast at 
the pixel for which the values are computed. Thus, the 
Sobel filtering procedure shown in Fig. 13 generates 
values representative of the contrast in the image at 
each respective pixel. 

Referring to Fig. 15, the procedure of Fig. 12 
equalizes 76 the results of the Sobel filtering proce- 
dure discussed above. To accomplish this, for each 
pixel in the first buffer 90 the computing circuitry re- 
verses 92 black and white values in the manner dis- 
cussed above, replaces 94 each luminance value in 
the first buffer with the logarithm of the luminance val- 
ue, and then reverses 96 black and white values. 

The effect of the logarithmic computation descri- 
bed above is illustrated in Figs. 16A and 16B. As 
shown in Fig. 16A, a histogram of all of the Sobel-f ilter 
results in the first buffer after step 74 includes a large 
number of pixels with near zero values and a large 
number of pixels with values at the maximum permis- 
sible value. However, in the mid-range of values, 
there is a relatively low number of pixels. However, as 
seen in Fig. 16B, by replacing each value with the log- 
arithm of the value, the number of pixels in the mid- 
range of luminance values is substantially increased. 
Thus, the logarithmic histogram equalization increas- 
es, in a non-linear manner, small Sobel-filter result 
values to larger values. Therefore, effect of this oper- 
ation on the image is to increase the emphasis on 
edges in images. 

It should be noted that, for the logarithmic equal- 
ization to operate properly, areas of high contrast 
must be represented by large luminance values, and 
other areas represented by small luminance values. 



However, in step 85 (Fig. 13) black and white are re- 
versed so that contrast edges are represented by 
small luminance values. Therefore, black and white 
must be re-reversed in step 92 before the logarithmic 

5 computation. 

These two reversals of black and white in steps 
85 and 92 happen in immediate succession and may 
therefore initially appear to be unnecessary. How- 
ever, the two reversals are included so that the histo- 

10 gram equalization procedure, i.e., alt of the process- 
ing illustrated in Fig. 15, can be disabled at any given 
time without modifying any other procedures. This 
might be done to, for example, examine the differ- 
ence in results with and without the logarithmic oper- 

15 ation. In another application, steps 85 and 92 could be 
eliminated. 

Referring to Fig. 17, to combine 78 the processed 
pixels from the first buffer and the unprocessed pixels 
from the second buffer, the circuitry or software per- 

20 forms 1 00 a number of steps for each pixel in the im- 
age. First, the computing circuitry determines 102 
whether the unprocessed pixel in the second buffer 
has a luminance greater than the average luminance 
of the central one-third of the image (as previously 

25 computed) and whether the processed pixel in the 
first buffer is less than midscale of the possible lumi- 
nance values. If both are the case, the computing cir- 
cuitry reverses 104 black and white values in the first 
buffer in the manner discussed above. This causes 

30 edges in relatively light-colored areas to have bright 
outlines, while causing edges in relatively dark- 
colored areas to have dark outlines. Then, the com- 
puting circuitry produces a final luminance value for 
the pixel by multiplying 106 the corresponding pixels 

35 from the first and second buffer with each other and 
with a gain value. The resulting luminance value is 
limited 108 to the allowable range for luminance val- 
ues and is stored as the final luminance value for the 
pixel. 

40 Referring to Fig. 18, to smooth 80 the image pro- 

duced by the proceeding steps, the circuitry or soft- 
ware performs 110 a loop processing each pixel by 
averaging 112 the pixel with a neighboring pixel, e.g., 
the pixel vertically neighboring the pixel. The result of 

45 this operation is to slightly blur the resulting image 
and produce a more realistic watercolor effect. 

While the present invention has been illustrated 
by a description of various embodiments and while 
these embodiments have been described in consid- 

50 enable detail, it is not the intention of the applicants 
to restrict or in any way limit the scope of the append- 
ed claims to such detail. Additional advantages and 
modifications will readily appear to those skilled in 
the art. For example, the procedures described above 

55 use the average luminance value from the central 
third of the image to approximate the average lumi- 
nance value across the image. It has been found that 
this technique can result in erroneous results if the 
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central third of the image is particularly bngh or d ark 
Thus in one alternative embodiment multaple Affer- 
ent areas of the image may be used to oomputej. 
arageluminancevalue.Theinventionm.sbr^dj 

representative apparatus and method, andjHustra 
tave example shown and described. Accordingly, de 
oartures may be made from such details without d» 
Tarting from the scope of applicant's genera, mven- ^ 
tive concept. 



Claims 

i A method for modifying an initial image to pro- 
duce a resulting image having the appearance of 
custom art work, comprising 

storing a pixellated representation of sa.d 
initial image in an electronic n*™"*' 

inspecting pixels of sa.d pixe "ated repre- 
sentation of said initial image to denve. without 

cisTon of said initial image in a manner s.mula tang 
techniques used by humans to produce custom 

Automatically applying said modifications 
to said pixels of said pixellated representation o 
said initial image to generate a pixellated repre 
sentation of said resulting image, and 

producing a visual output of said resulting 

image. 

2 The method of claim 1 wherein producing a visual 
output of said resulting image comprises one or 

^ printing a copy of said pixellated represen- 
tation of said resulting image and 

generating a display of said pixellated rep- 
resentation of said resulting image on an elec- 
tronic display device. 
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to an average luminance value denved from a 
Dlurality of pixels of said initial image, 
' reducing said selected luminance value .f 
said selected luminance value is less than said 
average luminance value, otherw.se .ncreas.ng 

<;aid selected luminance value, 

deriving pixel luminance and color values 

from said selected color value and sa.d reduced 

or increased selected luminance value, and 

modifying pixels of said selected portion 

to approach said derived pixel luminance and col- 

or values. 

5. The method* daim 4 wherein deriving pixel lu- 
minance and color values comprises augmen t, g 
said selected color value and sa.d reduced o in- 
cased selected luminance value with pseudo- 
random noise to produce variation ,n luminance 
or color of pixels of said selected portion. 

G The method of claim 3 wherein selecting portions 
If saTd mitia. image comprises selecting elongat- 
ed Portions having substantially parabolic curved 
elongated edges. 

7 The method of claim 3 wherein selecting portions 
of said initial image further comprises selecting 
e.ongated portions oriented in one of at least two 
possible directions. 

8 The method of claim 7 wherein selecting portions 
of said initial image further compnses 

selecting a location for a portion, 
determining contrast of said in.t.al image 
in each of four directions at said location, and de- 
rerlgadirectionofmaximumcontrastatsaid 

'^selecting an elongated portion at said lo- 
cation said elongated portion being oriented with 
40 its elongated dimension perpendicular to sa.d d- 
rectton of maximum contrast. 



30 



3 The method of claim 1 wherein modifying pixels 
of saw pixellated representation of said initial .m- 

a9e ^ect; S gportionsofsaidinitia.image.and « 

modifying the selected portions to approx- 
imate one or more of uniform luminance, uniform 
Sot saturation and uniform color hue as com- 
pared to said initial image, 

whereby said resulting image has the ap- so 
pearance of brush strokes. 

4. The method of claim 3 wherein modifying a se- 
lected portion comprises 

selecting a luminance and color value cor 
responding to one or more pixels within said se- 
lected portion of said initial image. 

comparing said selected luminance value 



g. Themethodofclaim8where.nselect,ngp^,ons 
of said initial image further compnses se lectang 
an elongated portion having a size related to con- 
trast of said initial image in said direction of max- 
imum contrast. 

1 o The method of claim 8 wherein determining con- 
trast of said initial image comprises determining 
contrast from a blurred version of sa.d .nitaal im- 
age, resulting in increased uniformity of onenta- 
tion of said portions. 

11. The method of claim 3 wherein reducing the pre- 
cision of said initial image further compnses blur- 
?ing said initial image to form a background w.th,n 
which said portions are modified. 



15 



EP 0 678 831 A2 



16 



12. The method of claim 3 wherein 

selecting portions of said initial image 
comprises selecting a f irst set of rows of blocks 
extending through said image, and then selecting 
a second set of rows of blocks interleaved with 
said first set of rows of blocks and extending 
through said image, and 

modifying said portions comprises modi- 
fying portions along said first set of rows prior to 
modifying portions along said second set of rows, 

whereby modifications of said portions 
are variably overlapped within said image. 

13. The method of claim 3 wherein 

selecting portions of said initial image 
comprises selecting large portions and smaller 
portions of said image, and 

modifying said portions comprises modi- 
fying said large portions prior to modifying said 
smaller portions. 

14. The method of claim 13 wherein 

selecting large portions of said image 
comprises selecting large portions along a first 
set of rows extending through said image, and se- 
lecting large portions along a second set of rows 
interleaved with said first set of rows and extend- 
ing through said image, and 

modifying said portions comprises modi- 
fying portions along said first set of rows prior to 
modifying portions along said second set of rows, 

whereby modifications of said large por- 
tions are variably overlapped within said image. 

15. The method of claim 1 3 wherein 

selecting smaller portions of said image compris- 
es determining contrast of said image and select- 
ing smaller portions in regions of said image hav- 
ing high contrast. 

16. The method of claim 1 wherein 

modifying pixels of said pixellated representation 
of said initial image to reduce the precision of said 
initial image comprises 

locating contrast edges in the initial im- 
age, and 

modifying luminance of said initial image 
along said edges, whereby objects in said result- 
ing image have enhanced definition from back- 
ground. 

17. The method of claim 16 wherein 

modifying luminance of said initial image along 
said edges comprises increasing luminance of 
said image along edges in relatively high lumi- 
nance areas of said image, and decreasing lumi- 
nance of said image along edges in relatively low 
luminance areas of said image. 



18. The method of claim 16 wherein 

modifying luminance of said initial image along 
said edges comprises 

generating at each pixel of said initial im- 

5 age a differential measure of luminance contrast 

in two directions, and 

modifying luminance of pixels of said ini- 
tial image by combining each said differential 
measure with luminance of a corresponding pixel 

10 in said initial image, to generate luminance val- 
ues for said resulting image. 

19. The method of claim 18 wherein 

generating a differential measure of contrast at a 
15 pixel includes computing a luminance variation 

between pixels neighboring said pixel. 

20. The method of claim 19 wherein 

generating a differential measure of contrast at a 
20 pixel includes computing a logarithm of said lumi- 

nance variation to enhance sensitivity to contrast 
edges in said initial image. 

21. The method of claim 18 wherein combining each 
25 said differential measure with its corresponding 

pixel in said initial image comprises multiplying 
each said differential measure with luminance of 
a corresponding pixel in said initial image. 

30 22. The method of claim 1 8 wherein 

combining each said differential measure with a 
corresponding pixel in said initial image compris- 
es 

determining whether said differential 
35 measure is below a threshold value, and if so 

modifying said differential measure by subtract- 
ing it from a predetermined maximum value, and 
multiplying each thus modified differential 
measure with luminance of said corresponding 
40 pixel in said initial image. 

23. The method of claim 18 wherein 

combining each said differential measure with a 
corresponding pixel in said initial image compris- 
45 es 

deriving an average luminance value for 
said initial image from a plurality of pixels of said 
initial image, 

determining whether luminance of said 
50 corresponding pixel in said initial image is above 
said average luminance value, and if so modify- 
ing said differential measure by subtracting it 
from a predetermined maximum value, and 

multiplying each thus modified differential 
55 measure with luminance of said corresponding 

pixel in said initial image. 

24. Apparatus for modifying an initial image to pro- 
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duce a resulting image having the appearance of 
custom art work, comprising 

an electronic memory storing a pixellated 
representation of said initial image, 

computing circuitry inspecting pixels of 5 
said pixellated representation of said initial image 
to derive, without human input, modifications 
which reduce the precision of said initial image in 
a manner simulating techniques used by humans 
to produce custom art work, and automatically 10 
applying said modifications to said pixels of said 
pixellated representation of said initial image to 
generate a pixellated representation of said re- 
sulting image, and . 32. 

an output port for connection to an output 1 5 
device capable of producing a visual output of 
said resulting image. 

25. The apparatus of claim 24 wherein said output 
port is configured for connection to a printer. 20 33. 

26. The apparatus of claim 24 wherein said output 
port is configured for connection to an electronic 
display monitor. 

25 

27. The apparatus of claim 24 wherein said comput- 34. 
ing circuitry modifies pixels of said pixellated rep- 
resentation of said initial image by selecting por- 
tions of said initial image and modifying the se- 
lected portions to approximate one or more of 30 
uniform luminance, uniform color saturation and 
uniform color hue as compared to said initial im- 
age, 

whereby said resulting image has the ap- 
pearance of brush strokes. 35 35. 

28. The apparatus of claim 27 wherein said comput- 
ing circuitry selects portions of said initial image 
by selecting a location for a portion, determining 
contrast of said initial image in each of four direc- 40 
tions at said location, determining a direction of 
maximum contrast at said location, and then se- 
lecting an elongated portion at said location, said 36 
elongated portion being oriented with its elongat- 
ed dimension perpendicular to said direction of 45 
maximum contrast. 

29. The apparatus of claim 28 wherein elongated 
portions selected by said computing circuitry 
have a size related to contrast of said initial image so 
in said direction of maximum contrast. 



30. The apparatus of claim 28 wherein said comput- 
ing circuit determines contrast of said initial im- 
age from a blurred version of said initial image, 
resulting in increased uniformity of orientation of 
said portions. 



31. The apparatus of claim 27 wherein 

said computing circuit selects portions of 
said initial image by selecting a first set of rows 
of blocks extending through said image, and then 
selecting a second set of rows of blocks inter- 
leaved with said first set of rows of blocks and ex- 
tending through said image, and 

said computing circuit modifies portions 
along said first set of rows prior to modifying por- 
tions along said second set of rows, 

whereby modifications of said portions 
are variably overlapped within said image. 

The apparatus of claim 27 wherein 

said computing circuit selects large por- 
tions and smaller portions of said image, and 

said computing circuit modifies said large 
portions prior to modifying said smaller portions. 

The apparatus of claim 32 wherein said comput- 
ing circuit selects smaller portions of said image 
by determining contrast of said image and select- 
ing smaller portions in regions of said image hav- 
ing high contrast. 

The apparatus of claim 24 wherein said comput- 
ing circuitry modifies pixels of said pixellated rep- 
resentation of said initial image to reduce the pre- 
cision of said initial image by locating contrast 
edges in the initial image, and modifying lumi- 
nance of said initial image along said edges, 
whereby objects in said resulting image have en- 
hanced definition from background. 

The apparatus of claim 34 wherein said comput- 
ing circuitry modifies luminance of said initial im- 
age along said edges by increasing luminance of 
said image along edges in relatively high lumi- 
nance areas of said image, and decreasing lumi- 
nance of said image along edges in relatively low 
luminance areas of said image. 

The apparatus of claim 34 wherein said comput- 
ing circuitry modifies luminance of said initial im- 
age along said edges by generating at each pixel 
of said initial image a differential measure of lu- 
minance contrast in two directions, and modify- 
ing luminance of pixels of said initial image by 
combining each said differential measure with lu- 
minance of a corresponding pixel in said initial im- 
age, to generate luminance values for said result- 
ing image. 

37. The apparatus of claim 36 wherein said comput- 
55 ing circuitry generates a differential measure of 

contrast at a pixel by computing a luminance va- 
riation between pixels neighboring said pixel. 
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38. The apparatus of claim 37 wherein said comput- 
ing circuitry generates a differential measure of 
contrast at a pixel by computing a logarithm of 
said luminance variation, thereby enhancing sen- 
sitivity to contrast edges in said initial image. 

39. The apparatus of claim 36 wherein said comput- 
ing circuitry combines each said differential 
measure with its corresponding pixel in said initial 
image by multiplying each said differential meas- w 
ure with luminance of a corresponding pixel in 
said initial image. 
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have more uniform luminance, color saturation, or color 
hue, producing a resulting image appearing to contain 
brush strokes. Alternatively, the precision of the initial 
image is reduced by locating contrast edges in the im- 
age, and modifying the luminance along these edges, 
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tures are outlined and have an enhanced definition from 
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